Papers with SIFo tasks
The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models (2024.findings-emnlp)
Copied to clipboard
Xinyi Chen, Baohao Liao, Jirui Qi, Panagiotis Eustratiadis, Christof Monz, Arianna Bisazza, Maarten Rijke
| Challenge: | Current evaluation resources for instruction following focus on single task instructions, but the instruction sequences in these benchmarks often lack coherence. |
| Approach: | They propose to evaluate models’ abilities to follow multiple instructions through sequential instruction following tasks using four tasks to assess different aspects of sequential instruction followed. |
| Outcome: | The proposed benchmark outperforms open-source and closed-source models on four tasks assessing different aspects of sequential instruction following. |